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Program description 



Caring School Community™ (CSC) is a modified version of a 
program formerly known as the Child Development Project. 

The Caring School Community™ program has been recently 
revised to eliminate some elements of the Child Development 
Project that were shown in evaluation studies to be incon- 
sistently or poorly implemented. CSC is a multiyear school 
improvement program that involves all students in grades K-6. 
The program aims to promote core values, prosocial behavior, 
and a schoolwide feeling of community. The program consists 
of four elements originally developed for the Child Development 
Project: class meeting lessons, cross-age “buddies” programs, 
“homeside” activities, and schoolwide community. Class 
lessons provide teachers and students with a forum to get to 



know one another, discuss issues, identify and solve problems 
collaboratively, and make a range of decisions that affect 
classroom life. Cross-age buddies activities pair whole classes 
of older and younger students for academic and recreational 
activities that build caring cross-age relationships and cre- 
ate a schoolwide climate of trust. Homeside activities, short 
conversational activities that are sent home with students for 
them to do with their parent or caregiver and then to discuss 
back in their classroom, incorporate the families’ perspectives, 
cultures, and traditions, thereby promoting interpersonal 
understanding. Schoolwide community-building activities bring 
students, parents, and school staff together to create new 
school traditions. 



Research Two studies of the Child Development Project program met the 5,600 students from grades K-6 in seven school districts across 
What Works Clearinghouse (WWC) evidence standards with the country. 1 
reservations. The two studies together included more than 



1. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 
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The Child Development Project was found to have potentially positive effects on behavior; no discernible effects on knowledge, 
attitudes, and values; and no discernible effects on academic achievement. 



Behavior 


Knowledge, attitudes, 
and values 


Academic achievement 


Potentially positive effects 


No discernible effects 


No discernible effects 



Rating of effectiveness 
Improvement index 2 



Average: +8 percentile points 
Range: -2 to +27 percentile 
points 



Average: +7 percentile points 
Range: -4 to +18 percentile 
points 



Average: +4 percentile points 
Range: -8 to +16 percentile 
points 



Developer and contact 

Distributed by Developmental Studies Center, 2000 Embarcadero, 
Suite 305, Oakland, CA 94606. Program contact: Denise Wood. 
E-mail: info@devstu.org. Web: www.devstu.org/csrd/cdp index, 
html . Phone: (800) 666-7270 ext. 239. Fax: (510) 464-3670. 

Scope of use 

In the fall of 2004, the updated intervention, Caring School 
Community™, replaced the earlier intervention, the Child 
Development Project (CDP). The two interventions are reviewed 
together in this report, under the name CSC, because all CSC 
elements were part of CDP. 3 The CDP was implemented in 321 
schools across the nation. Since its release in the fall of 2004, 
CSC has been implemented in 2,756 classrooms. Information 
is not available on the number or demographics of students, 
schools, or districts using the intervention. The studies reviewed 
implemented the CDP rather than the CSC program. The 
WWC recommends asking the developer for information about 
the most current version of the CSC program and taking into 
account that student demographics and school context may 
affect outcomes. 



Teaching 

The program was developed based on research-supported 
claims that students’ academic, social, and ethical develop- 
ment benefit from: caring school communities; having their 
psychological needs for autonomy, belonging, and competence 
met; having a better sense of “connectedness” to schools (that 
is, students like school, have trust and respect for teachers, and 
have high educational aspirations); cooperative rather than com- 
petitive learning environments; and social support and guidance 
from teachers in formal and informal learning situations. 

All four components of CSC — class meeting lessons, cross- 
age buddies programs, homeside activities, and schoolwide 
community — are designed to be introduced over the course of 
one year. However, according to the developer, some schools 
may decide to introduce the components more gradually. 

Class lessons are designed to teach core values, including 
fairness, helpfulness, caring, respect, and personal responsibil- 
ity. A typical session uses a scripted lesson and begins with 
a brief review of the class meeting rules (established norms to 
maintain a sense of community in the classroom) collaboratively 
established by the teacher and students. Then, the teacher 



2. These numbers show the average and the range of improvement indices for all findings across the two studies. 

3. According to the developer, what was dropped from CDP in creating CSC were two other program elements (a classroom management program called 
“Developmental Discipline” and a literature-based reading program called “Reading, Thinking, and Caring” at grades K-3 and “Reading for Real” at grades 
4-6) that evaluation studies conducted from the early 1980s through the mid-1990s found (through third-party classroom observations and teacher self- 
reports) to be too difficult for many teachers to implement. 
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introduces the topic of discussion for the lesson. During the 
discussion, the teacher encourages students to maintain positive 
interpersonal communication in which they build on each other’s 
thinking. Sometimes the discussion is followed by activities 
done individually or in groups. The teacher concludes the lesson 
by summarizing what was done during the lesson and setting 
expectations regarding students’ future behavior. 

A schoolwide component of CSC, cross-age buddies activi- 
ties, involves older students mentoring younger students for 
academic activities, cooperative learning skills, and relationship 
building. The buddies activities require one hour of class time 
a week or month, and an additional 15 minutes of teacher 
preparation time. Students participate in additional schoolwide 
activities such as interviewing nonteaching staff members, 
holding a family projects fair, and planting a school community 
garden. Homeside activities are included in the curriculum to 
build positive relationships between home and school and honor 
what families and communities have to offer. These activities 
may include show and tell from home, sharing a holiday tradition, 
or telling a family folklore story. Homeside activities are available 
in English and Spanish. CSC read-aloud libraries are an optional 
program enhancement. According to the developer, reading and 
discussing these books provide opportunities for students to 
make connections between the values they are learning in the 
CSC program and the social and ethical values in the literature. 

Teachers participate in a one-day workshop in which they 
explore the program components, discuss ways to build caring 



and supportive environments, learn how to facilitate student 
conversations, learn strategies to enhance cooperation among 
students, explore strategies to build community within and 
across classrooms, and view and discuss video vignettes of 
classroom practices. Additional coaching for teachers is avail- 
able. A training of trainers approach is also available. According 
to this approach, a team of four to six participants from a school 
receives three-day institute training and then provides staff 
development to the remaining school staff. District-level coaches 
and staff developers can participate in an array of professional 
development offerings. 

Cost 

A classroom package that contains class meeting lessons, 
teacher’s calendar, cross-age buddies activity book, homeside 
activities, and schoolwide community-building activities costs 
$185. The principal’s package (which includes all classroom 
materials for teachers plus a principal’s leadership guide) costs 
$275. The cost of workshops and follow-up visits is $2,000 a 
day, plus travel expenses. Total cost for training varies depend- 
ing on the number of professional development days needed 
and whether adoption involves a single school, multiple schools, 
or training-of-trainers at the district level. An optional enhance- 
ment to the program is the Caring School Community™ read- 
aloud libraries; the cost of the individual grade-level libraries 
ranges from $52-$67 a grade, and the cost for a complete K-6 
library is $408. 



Two studies reviewed by the WWC investigated the effects of the 
CDP, an earlier version of CSC. The studies are referred to as the 
“San Ramon Study” and the “Six-District Study” because there 
are many documents for each study. The San Ramon Study 



was a randomized controlled trial that met WWC standards 
with reservations. 4 The Six-District Study used a quasi- 
experimental design that also met WWC evidence standards 
with reservations. 



4. The San Ramon Study randomly assigned one group of three schools to intervention or comparison groups. Because the unit of assignment consisted 
of one set of schools, there is confounding between the unit of assignment and the unit of intervention. The study authors collected baseline measures 
and demonstrated that the intervention and comparison schools were matched at baseline in terms of relevant students’ outcomes. Therefore, although 
this study did not meet WWC standards as a randomized controlled trial because of the confounding effect, it met standards with reservations as a quasi- 
experimental design. 
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The San Ramon Study included more than 600 students a 
year from two cohorts 5 of grades K-4 in 27 classrooms in six 
elementary schools serving a middle- to upper-class suburban 
community in San Ramon, California. Outcomes for students 
in the three schools using CDP were compared to those for 
students in three matched comparison schools. The Six-District 



Study assessed approximately 5,500 grades 3-6 students 
each year for four consecutive years in 24 elementary schools 
in six school districts across the United States. 6 Outcomes for 
students in schools using CDP were compared with those for 
students in matched comparison schools. Both studies focused 
on CDP implemented as a schoolwide intervention. 



Findings 

The WWC review of interventions for character education 
addresses student outcomes in three domains: behavior; 
knowledge, attitudes, and values; and academic achievement. 

Behavior. The San Ramon Study examined four types of 
student behavior in the classroom and found statistically sig- 
nificant positive effects on two of them (spontaneous prosocial 
behavior and supportive, friendly and helpful behavior). 7 The 
WWC confirmed the statistical significance of both findings. 8 The 
Six-District Study examined students’ altruistic behavior and 19 
different indicators of problem behavior and victimization. The 
study reported statistically significant effects for two outcomes 
(use of alcohol and use of marijuana). However, none of these 
outcomes were statistically significant as calculated by the WWC. 
Additionally, the average effect size across all findings in the 
behavior domain was not large enough to be considered sub- 
stantively important by WWC standards (that is, at least 0.25). So, 
the Six-District Study showed indeterminate effects on behavior. 

Knowledge, attitudes, and values. The San Ramon Study 
examined values, attitudes, and relevant social skills and sense 
of school as a community of elementary school students. The 



study reported statistically significant positive effects on 
several outcomes, including social problem-solving skills and 
conflict resolution skills, democratic values, consideration of 
others’ needs, and sense of community. 9 However, none of 
these outcomes were statistically significant as calculated by 
the WWC. The average effect size across all findings in this 
domain was neither statistically significant nor large enough to 
be considered substantively important by WWC criteria (that is, 
at least 0.25). The Six-District Study examined core values, atti- 
tudes, and related social-emotional skills of elementary school 
students and reported statistically significant positive effects 
on several outcomes, including concern for others, democratic 
values, conflict resolution skills, outgroup discrepancy score 
(deviation from friend), liking for school, enjoyment of class, 
task orientation, and sense of community. 10 However, none of 
these outcomes were statistically significant as calculated by 
the WWC. 8 The average effect size across all findings in this 
domain was neither statistically significant nor large enough to 
be considered substantively important by WWC criteria (that is, 
at least 0.25). So, both studies showed indeterminate effects on 
knowledge, attitudes, and values. 



5. The first cohort was tracked for five years and the second cohort for two years. 

6. This review includes only five intervention schools with meaningful progress toward program implementation and their matched comparison schools. 

7. This analysis was reported by Solomon et al. (1988). 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted 
Computations for the formulas the WWC used to calculate the statistical significance. In the case of CSC/CDP, corrections for clustering and multiple 
comparisons were needed. 

9. These analyses were reported by Battistich et al. (1989), Benninga et al. (1991), Solomon et al. (1996), and in information provided to the WWC by the first 
study author. 

10. These analyses were reported by Battistich et al. (2000), Solomon et al. (2000), and in information provided to the WWC by the first study author. 
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Academic achievement. The San Ramon Study examined 
elementary school students’ academic performance and 
reported one statistically significant positive effect on a holistic 
measure of reading comprehension. However, this effect was 
not statistically significant as calculated by the WWC. 8 In addi- 
tion, the average effect size across all findings in the academic 
achievement domain was neither statistically significant nor 
large enough to be considered substantively important by 
WWC criteria (that is, at least 0.25). The Six District Study 
examined elementary school students’ academic motivation 
and performance. It reported statistically significant positive 
effects on intrinsic academic motivation and achievement in 
multiple state standardized tests. However, these effects were 
not statistically significant as calculated by the WWC. 9 In addi- 
tion, the average effect size across all findings in the academic 



achievement domain was neither statistically significant nor 
large enough to be considered substantively important by 
WWC criteria (that is, at least 0.25). So, both studies showed 
indeterminate effects. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as: positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings (as calculated 
by the WWC 8 ), the size of the difference between participants in 
the intervention condition and the comparison condition, and the 
consistency in findings across studies (see the WWC Interven- 
tion Rating Scheme) . 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technical 
Details of WWC-Conducted Computations) . The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition versus the 
percentile rank of the average student in the comparison condi- 
tion. Unlike the rating of effectiveness, the improvement index is 
entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results. The aver- 
age improvement index for behavior is +8 percentile points, with 



a range of -2 to +27 percentile points. The average improvement 
index for knowledge, attitudes, and values is +7 percentile 
points, with a range of -4 to +18 percentile points. The average 
improvement index for academic achievement is +4 percentile 
points, with a range of -8 to +16 percentile points. 

Summary 

The WWC reviewed two studies on CDP, a former version of the 
CSC. Both studies met WWC evidence standards with reserva- 
tions. The WWC analysis found potentially positive effects in the 
behavior domain and no discernible effects in the knowledge, 
attitudes, and values and the academic achievement domains. 
Character education, an evolving field, is beginning to establish 
a research base. The evidence presented in this report is limited 
and may change as new research emerges. 
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Appendix 



Appendix A1.1 Study characteristics: The San Ramon Study (randomized controlled trial with confounding problems) 1 



Characteristic 


Description 


Study citation 


Battistich, V., Solomon, D., Watson, M., Solomon, J, & Schaps, E. (1989). Effects of an elementary program to enhance prosocial behavior on children's cognitive-social 
problem-solving skills and strategies. Journal of Applied Developmental Psychology, 10(2), 147-169. 


Participants 


The study included two cohorts of elementary school students from six elementary schools. Each cohort began with the project in kindergarten. The first cohort started 
kindergarten in 1982-83 and the second in 1985-86. The first cohort included 13 kindergarten classrooms that were followed through the elementary school years; 2 the 
second cohort included 14 classrooms that were followed through the first grade. There were about 350 students a year in the first cohort (divided evenly between the 
intervention and comparison groups). Of those 350 students, about 165 students remained with the cohort all five years. 


Setting 


The study took place in six elementary schools located in a middle- to upper middle-class suburban community in northern California. 


Intervention 


The intervention schools implemented the Child Development Project (CDP) proaram, (For details about the connection between the CDP and the CSC, see the CSCinterven- 
tion report). 3 Students in the intervention aroup received the CDP proaram everv vear startina in kinderaarten. Class meetinas in the intervention condition included activities 
designed to promote core values. In the classroom, students learned group interaction skills and relevant values and worked in small groups toward mutual academic and 
nonacademic goals. Teachers identified and discussed exemplary behavior using examples from the classroom, television, literature, and movies. Developmental discipline, a 
classroom management approach, was applied to teach prosocial norms and values. In addition, children were encouraged to help others by doing classroom chores, tutoring 
younger students as part of the “buddies” programs, performing charitable community activities, and helping with activities in the school at large. An implementation check 
done by two independent observers indicated a high level of implementation and significantly different classroom experiences (with respect to classroom activities and teacher 
behavior) in the intervention classrooms compared with the comparison classrooms. 


Comparison 


The comparison group included three elementary schools in the same school district as the intervention schools and matched with the intervention schools on socioeconomic 
status and interest in the intervention. Comparison group students did not participate in the Caring School Community program. No information was provided on character 
education related practices in the comparison schools. 


Primary outcomes 
and measurement 


Student outcomes in three domains were examined: behavior; knowledge, attitudes, and values; and academic achievement. Students’ behavior was assessed using direct 
observations of students’ behavior in the classroom. Students’ knowledge, attitudes, and values were assessed using several self-report questionnaires. Academic achieve- 
ment was assessed using standardized achievement tests. (See Appendices A2.1-A2.3 for more detailed descriptions of the outcome measures.) 


Teacher training 


Teacher training consisted of a one-week summer institute, monthly workshops, frequent meetings with project staff who also observed the classrooms periodically, and 
supporting curriculum materials. 



1 . The San Ramon Study randomly assigned one group of three schools to intervention or comparison groups. Because the unit of assignment consisted of one set of schools, there is confounding 
between the unit of assignment and the unit of intervention. The study authors collected baseline measures and demonstrated that the intervention and comparison schools were well matched in 
terms of relevant students’ outcomes. Therefore, although this study did not meet WWC standards as a randomized controlled trial because of the confounding effect, it met standards as a quasi- 
experimental design. 

2. The study authors also conducted follow-up analyses to examine student outcomes in grades 5 and 6 (Battistich, 2003; Solomon, Battistich, & Watson, 1993; Solomon, Watson, Battistich, 
Schaps, & Delucchi, 1996); however, in grade 5 only one school per condition participated in the study, therefore, due to confounding of the school with the intervention, this analysis was not 
reviewed. In addition, in grade 6, only two schools per condition participated in the analysis; therefore, due to severe attrition on the school level, and small percentage of students receiving the 
program since kindergarten (35%) this follow-up analysis was not reviewed. 

3. According to Battistich (2003), this intervention has recently been modified in a significant way. Therefore, this study presents an assessment of the impact of the intervention as it was configured 
at the time of the study. 
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Appendix A1.2 Study characteristics: The Six-District Study (quasi-experimental design) 



Characteristic 


Description 


Study citation 


Battistich, V., Schaps, E., Watson, M., Solomon, D., & Lewis, C. (2000). Effects of the Child Development Project on students’ drug use and other problem behaviors. Journal 
of Primary Prevention, 27(1), 75-99. 


Participants 


Participants of the study were students in the upper elementary grades in 12 intervention schools and 12 matched comparison schools in six districts (grades 3-5 in four 
districts and grades 4-6 in the two other districts). This review includes only five intervention schools with meaningful progress toward program implementation and their 
matched comparison schools. 1 The composition of the student population was similar at the intervention and comparison schools. Two of the schools in the sample reviewed 
and their matched comparison schools served a predominantly low-socioeconomic status population. In four pairs of schools, most of the school population was white; in one 
pair of schools, most of the students were African-American. The students began with the study in 1991-92 when they were in the third or fourth grade and were followed 
until the end of elementary school. 2 


Setting 


The study took place in 24 schools located in six urban, suburban, and rural districts and serving diverse student populations. The 10 schools included in this review were 
from three districts: one district on the West Coast, one in the South, and one in the Southeast. Two schools (one intervention and one comparison) were from a rural school 
district. Four schools (two intervention and two comparison) were located in an urban school district, and four schools (two intervention and two comparison) were located in a 
suburban school district. 


Intervention 


The intervention schools implemented the Child Development Project (CDP) oroaram. (For details about the connection between the CDP and the CSC, see the CSC intervention 
report) . The CDP oroaram consisted of classroom discussions and activities, a schoolwide component, and a family involvement component. Class meetinas included activities 
designed to promote core values. In the classrooms, students learned group interaction skills and relevant values and worked in small groups toward mutual academic and 
nonacademic goals. Teachers identified and discussed exemplary behavior using examples from the classroom, television, literature, and movies. Developmental discipline, a 
classroom management approach, was applied to teach prosocial norms and values. In addition, children were encouraged to help others by doing classroom chores, tutoring 
younger students as part of the “buddies” programs, performing charitable community activities, and helping with activities in the school at large. Classroom observations and 
interviews with school staff indicated an adequate level of program implementation. 


Comparison 


The comparison schools were drawn from the same school districts as the intervention schools and matched with the intervention schools with respect to school size and 
student characteristics. The comparison schools did not implement the program. 


Primary outcomes 
and measurement 


The study investigated students’ drug use and other types of problem behavior, core values (acceptance of people in outgroups, concern for others, altruistic behavior), and 
academic attitudes and motives (sense of the school as a community, task orientation, frequency of reading self-chosen books outside of school, frequency of reading self- 
chosen books in school, enjoyment of class, preference for challenging tasks). (See Appendices A2.1-A2.3 for more detailed descriptions of the outcome measures.) 


Teacher training 


Professional development was conducted at both the district and the school levels. At first, the program was introduced to 8-15 member “implementation teams” in each 
district. In the three subsequent years of the study, schoolwide training was also conducted. Each year, the implementation teams participated in summer workshops delivered 
by the developer. Implementation team members took increasing responsibility for the within-district workshops and for other support to teachers implementing the program. 
Teachers were also encouraged to meet regularly in small “partner study and support groups” to discuss and help each other with implementation issues. 



1 . The other seven intervention schools in four school districts did not demonstrate “meaningful progress towards implementation of the CDP program,” according to the study authors, with as 
many teachers in these schools showing no changes or even declines from baseline on a measure of program implementation as the number of teachers showing positive changes from baseline in 
implementation. Therefore, these seven intervention schools and their matched comparison schools were not included in this review. 

2. The study authors (Battistich, 2001; Battistich, Schaps, & Wilson, 2004) conducted additional follow-up analyses when the students attended middle schools. But these analyses did not identify 
those students from feeder schools that reached meaningful progress toward implementation. In addition, these analyses did not control for baseline differences between students in the interven- 
tion and comparison groups; therefore, they were not included in this review. 
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Appendix A2.1 Outcome measures in the behavior domain 



Outcome measure 


Description 


Negative behavior 


Students’ interpersonal negative behavior in the classroom. This is scale derived from a classroom observation sign-system instrument developed for the purposes of the 
study reported by Battistich et al. (1989). Segments of two-minute observations were coded using a prespecified rating system to indicate frequency and quality of various 
behaviors and activities. 


Spontaneous prosocial behavior 


Students’ spontaneous helpfulness, concern for others, and cooperation. This is a scale derived from a classroom observation sign-system instrument developed for the 
purposes of the study reported by Battistich et al. (1989). Segments of two-minute observations were coded using a prespecified rating system to indicate frequency and 
quality of various behaviors and activities. 


Harmoniousness 


Students’ harmoniousness, apparent interest and involvement, and apparent happiness. This is a scale derived from a classroom observation sign-system instrument 
developed for the purposes of the study reported by Battistich et al. (1989). Segments of two-minute observations were coded using a prespecified rating system to 
indicate frequency and quality of various behaviors and activities. 


Supportive, friendly, 
and helpful behavior 


Students’ supportive, friendly, and helpful behavior in the classroom, which may include one or more of the following behaviors: support and encouragement to other 
students, affection, inviting others to join activities, and thanking or praising other students. This is a scale derived from a classroom observation sign-system instrument 
developed for the purposes of the study reported by Battistich et al. (1989). Segments of two-minute observations were coded using a prespecified rating system to 
indicate frequency and quality of various behaviors and activities. 


Social competence 


Teacher’s ratings of students' social competence based on students' behavior (as cited in Solomon et. at, 1989). Four ratings were combined to form this general measure 
(takes an active role in resolving personal difficulties or problems; is admired and sought after by peers; does not expect others to provide for his or her every need; and 
gets along easily and comfortably with adults). Teachers rated each student on his or her social competence relative to other students in the class. 


Altruistic behavior 


A 10-item student self-report measure derived from Rushton, Chrisjohn, and Fekken (1981 ; as cited in Solomon et al., 2000) that assesses students' altruistic behavior. 


Use of alcohol 


A single survey item assessing lifetime use of alcohol (as cited in Battistich et al., 2000). 


Use of marijuana 


A single survey item assessing lifetime use of marijuana (as cited in Battistich et at, 2000). 


Use of cigarettes 


A single survey item assessing lifetime use of cigarettes (as cited in Battistich et al., 2000). 


Ran away from home 


A single survey item assessing frequency of involvement in one type of delinquent behavior: ran away from home during the past year (as cited in Battistich et al., 2000). 


Skipped school 


A single survey item assessing frequency of involvement in one type of delinquent behavior: skipped school during the past year (as cited in Battistich et al., 2000). 


Damaged property on purpose 


A single survey item assessing frequency of involvement in one type of delinquent behavior: damaged property on purpose during the past year (as cited in Battistich et al, 2000). 


Stolen money or property 


A single survey item assessing frequency of involvement in one type of delinquent behavior: stolen (or attempted to steal) money or property during the past year (as cited 
in Battistich et al, 2000). 


Carried a knife, gun, 
or other weapon 


A single survey item assessing frequency of involvement in one type of delinquent behavior: carried a knife, gun, or other weapon during the past year (as cited in Battistich 
et al, 2000). 


Threatened to hurt someone 


A single survey item assessing frequency of involvement in one type of delinquent behavior: threatened to hurt someone during the past year (as cited in Battistich et al, 2000). 



(continued) 
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Appendix A2.1 Outcome measures in the behavior domain (continued) 



Outcome measure 


Description 


Hurt someone on purpose 


A single survey item assessing frequency of involvement in one type of delinquent behavior: hurt someone on purpose during the past year (as cited in Battistich et at, 2000). 


Taken a car without permission 


A single survey item assessing frequency of involvement in one type of delinquent behavior: taken a car without permission during the past year (as cited in Battistich et at, 2000). 


Been in a gang fight 


A single survey item assessing frequency of involvement in one type of delinquent behavior: been in a gang fight during the past year (as cited in Battistich et al., 2000). 


Thrown objects at people 


A single survey item assessing frequency of involvement in one type of delinquent behavior: thrown objects at people during the past year (as cited in Battistich et al., 2000). 


Been made fun of or 
called names 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: been made fun of or called names during the past year (as cited in 
Battistich et al., 2000). 


Had property damaged 
on purpose 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: had property damaged on purpose during the past year (as cited in 
Battistich et al., 2000). 


Had property stolen from desk 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: had property stolen from desk during the past year (as cited in Battistich 
et al., 2000). 


Had money or property 
taken by force or threat 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: had money or property taken by force or threat during the past year (as 
cited in Battistich et al., 2000). 


Been threatened with harm 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: been threatened with harm during the past year (as cited in Battistich 
et al., 2000). 


Been physically attacked 1 


A single survey item assessing frequency of being the subject of one type of victimization behavior: been physically attacked during the past year (as cited in Battistich 
et al., 2000). 



1 . This victimization measure is an indicator of students’ problem behavior inside school. 

Appendix A2.2 Outcome measures in the knowledge, attitudes, and values domain 1 



Outcome measure 


Description 


Democratic values 


A total score of several democratic values subscales. The subscales measure students’ endorsement of statements favoring equality of representation and participation, 
willingness to compromise, and belief in one’s responsibility to state opinions, even if unpopular. 


Perceptual benevolence 


A self-report measure of the prosocial value of kindness to others (as cited in Benninga et al., 1992). 


Empathy 


An 11-item self-report paper-and-pencil measure of empathy that was adapted from Bryant (1982; as cited in Solomon et al., 1996). 


Concern for others 


This measure asks students to indicate their views regarding others' problems and whether they feel empathy toward others and their problems. This measure was adapted 
from Solomon and Kendall (1979; as cited in Solomon et al., 1996). 


Concern for equality 


A self-report measure that addresses students’ prosocial concern for equality of participation and outcomes for individuals in social situations (as cited in Benninga et al., 1992). 



(continued) 
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Appendix A2.2 Outcome measures in the knowledge, attitudes, and values domain (continued) 



Outcome measure 


Description 


Motive to help others learn 


A self-report measure that assesses students’ motivation in helping other students in academic learning (as cited in Benninga et at, 1991). 


Social understanding 


This six-item interview was adapted from Flapan (1968; as cited in Solomon et al., 1996). Students responded to a series of questions about scenes from a movie (Our 
Vines Have Tender Grapes; MGM, 1945) showing a series of interactions between two children and two adults. 


Conflict resolution 
interview: general conflict 
resolution strategy 


A score derived from an individually administered interview that presents students with three hypothetical conflict situations and prompts proposed conflict resolution strat- 
egies (as cited in Battistich et. al., 1989). This outcome addresses students' suggested strategies to resolve social problems (for example, using aggression, cooperation, 
or appealing to authority), taking into account who is favored by this strategy (self, other, or both) and the needs of self, other, or both. 


Social problem solving: 
interpersonal sensitivity 


A score derived from the Social Problem-Solving Analysis Measure (SPSAM) (Elias et al., 1978; as cited in Battistich et. al., 1989). This outcome addresses students’ 
understanding of social problem situations and awareness of the thoughts and feelings of the person involved in each situation. 


Social problem solving: means- 
ends cognitive problem solving 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ ability to plan specific steps toward resolution, 
consider alternative courses of action, and anticipate obstacles to and consequences of one’s action. 


Social problem solving: 
obstacle means-end 
cognitive problem solving 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ ability to plan specific steps toward resolution, 
consider alternative courses of action, and anticipate consequences and additional obstacles when challenged or confronted by an obstacle. 


Social problem solving: 
obstacle outcome expectancies 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ belief that a strategy they proposed as an 
amendment to previously challenged suggestions will lead to a successful resolution of the interpersonal problem. 


Social problem solving: 
obstacle problem 
resolution strategies 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students' suggested strategy to resolve social problems 
(for example, using aggression, cooperation, or appealing to authority) when their previously proposed strategies are challenged. 


Social problem solving: 
outcome expectancies 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ belief that certain actions will lead to a suc- 
cessful resolution of the interpersonal problem. 


Social problem solving: 
primary resolution strategies 


A score derived from the SPSAM by Elias etal. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ suggested strategy to resolve social problems (for 
example, using aggression, cooperation, or appealing to authority), take into account who is favored by this strategy (self, other, or both), and consider the needs of self, other, or both. 


Social problem solving: 
proportion antisocial strategies 


A score derived from the SPSAM, an individual interview by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ consideration and 
relative use of antisocial strategies such as an aggressive or disruptive action. 


Social problem solving: 
proportion prosocial strategies 


A score derived from the SPSAM by Elias et al. (1978; as cited in Battistich et. al., 1989). This outcome addresses students’ consideration and relative use of prosocial 
strategies such as cooperation and sharing, polite request, and discussion of the problem. 


Conflict resolution interview: 
proportion antisocial strategies 


A score derived from the conflict resolution interview (as cited in Battistich et. al., 1989). This individual interview presents students with three hypothetical conflict situa- 
tions and prompts proposed conflict resolution strategies. This outcome addresses students’ consideration and relative use of such antisocial strategies as an aggressive or 
disruptive action. 



(continued) 
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Appendix A2.2 Outcome measures in the knowledge, attitudes, and values domain 



Outcome measure 


Description 


Conflict resolution interview; 
proportion prosocial strategies 


A score derived from the conflict resolution interview (as cited in Battistich et al., 1989). This individual interview presents students with three hypothetical conflict situa- 
tions and prompts proposed conflict resolution strategies. This outcome addresses students’ consideration and relative use of prosocial strategies such as cooperation and 
sharing, polite request, and discussion of the problem. 


Conflict resolution interview; 
consideration of others’ needs 


A score derived from the conflict resolution interview (as cited in Battistich et al., 1989). This individual interview presents students with three hypothetical conflict situa- 
tions and prompts proposed conflict resolution strategies. This outcome addresses students’ consideration of the others’ needs as well as one’s own. 


Total self-esteem 


A self-report measure (as cited in Benninga et al., 1992; Solomon et al., 1996). For the third-grade level, this score includes general self-esteem (personal feelings of 
self-worth) and academic self-esteem. For the fourth-grade level, this score also includes social self-esteem. 


Sense of community 


A self-report measure of students’ sense of the classroom and the school as a community that was developed for the purposes of the study reported by Battistich et al. 
(1989), Solomon et al. (1992), and Solomon et al. (1996). Sense of community is conceptualized as encompassing two main student perceptions: that their behavior and 
their classmates’ behavior show that they care about and are supportive of one another and that they have an important role in classroom decisionmaking and direction. In 
the study reported by Battistich et al. (2000) sense of community is measured using a 38-item survey that is composed of three subscales: student autonomy and influ- 
ence in the classroom, classroom supportiveness, and school supportiveness. 2 


Acceptance of outgroups 


A 10-item student self-report measure that assesses students’ acceptance of outgroups (as cited in Solomon et al., 2000). Students indicated how much they would want 
to do a specific task with various other people (not including a close friend) differing in social distance. 


Outgroups discrepancy score 
(deviation from friend) 


A 10-item student self-report measure that assesses students’ acceptance of outgroups (as cited in Solomon et al., 2000). Students indicated how much they would want 
to do a specific task with a close friend and with various other people differing in social distance. The discrepancy score subtracts scores for the members of other groups 
from that for the friend. 


Enjoyment of helping 
others learn 


A five-item student self-report measure adapted from Deer et al. (1988) and Solomon, Watson, Battistich, Schaps, and Delucchi (1992; as cited in Solomon et al., 2000), 
which assesses students’ enjoyment of helping other students learn. 


Social competence 


A 10-item student self-report measure that assesses social competence (as cited in Solomon et al., 2000). 


General self-esteem 


A four-item student self-report measure adapted from Solomon and Kendall (1979; as cited in Solomon et al., 2000) that measures student general self-esteem. 


Conflict resolution skills 


An eight-item student self-report measure adapted from Battistich et al. (1989; as cited in Solomon et al., 2000). 


Sense of efficacy 


A 10-item student self report measure developed by Cowen, Work, Hightower, Wyman, Pancer, & Lotyczewski (1991 ; as cited by Solomon et al., 2000). 



1 . This appendix presents a brief description of all outcomes that are presented in Appendices A3.1 -A4.3. 

2. Students’ sense of autonomy in the classrooms may include elements of teacher behavior in addition to student behavior. The WWC did not obtain statistical information on the classroom sup- 
portiveness and school supportiveness subscales separately; therefore the composite score was reviewed. 
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Appendix A2.3 Outcome measures in the academic achievement domain 1 



Outcome measure 


Description 


California Achievement Test (CAT) 


A standardized test that measures achievement in reading, language, spelling, mathematics, study skills, science, and social studies (as cited in Solomon et al, 1992). 


Holistic measure of reading comprehension 


A measure developed by the Educational Testing Services that assesses the use of high-order thinking and the development of text understanding (as cited in 
Solomon et al., 1992). Students are asked to read two brief passages and then respond to general questions about the meaning of the passages. 


Inductive reasoning 


A cognitive ability test adapted from Ennis and Millman (1985; as cited in Battistich et al., 2000) that presents a series of questions to students about the 
implications of various clues for solving mysteries. 


Stanford Achievement Test (SAT9) 


A national standardized test that measures student achievement in reading, language, spelling, study skills, listening, mathematics, science, and social science. 


SRA Achievement Series 


Achievement series that includes two forms, Forms 1 and 2. This series was published by the Science Research Associates, Inc. This test is designed to assess 
broad areas of knowledge, general skills, and their application. 


Liking for reading 


One survey item that assesses students’ enjoyment of reading books (as cited in Solomon et al., 2000). 


Educational aspirations 


One survey item that asks students how far they would like to go in school. Students rate their predictions on a five-point scale that ranges from “go to high 
school, but not graduate” to “finish college.” 


Educational expectations 


One survey item that asks students how far they think they really will go in school. Students rate their predictions on a five-point scale that ranges from “go to 
high school, but not graduate” to “finish college.” 


Achievement motivation 


A seven-item self-report measure that was used for the study reported by Battistich et al. (1989). The measure was adapted from Weiner and Kukla (1970) and 
Solomon and Kendall (1979) and assesses achievement motivation at school (as cited in Solomon et al., 1996). 


Intrinsic academic motivation 


A measure based on a student survey that was used for the study reported by Battistich et al. (2000). This measure is a ratio of two scales: intrinsic and 
extrinsic academic scales. This scale asks students to indicate why and when they typically do academic work. The measure was adapted from Connell and 
Ryan (1987) and Deer, Solomon, Watson, and Solomon (1988; as cited in Battistich et al., 2000). 


Task orientation 


An eight-item self-report measure that assesses the tendency to feel most satisfied when school work is challenging and leads to improved understanding. This 
measure, used in the study reported by Battistich et al. (2000), was developed by Nicholls (1989; as cited in Solomon et al., 2000). 


Ego orientation 


A four-item self-report measure that assesses tendency to feel most satisfied when school work allows one to demonstrate better performance than other 
students. This measure, used in the study reported by Battistich et al. (2000), was developed by Nicholls (1989; as cited in Solomon et al., 2000). 


Work avoidance 


A five-item measure of a student’s tendency to feel most satisfied when work is easy. This measure, used in the study reported by Battistich et al. (2000), was 
developed by Nicholls (1989; as cited in Solomon et al., 2000). 


Preference for challenging tasks 


A five-item self-report measure that was adapted from Weiner and Kukla (1970) and Solomon and Kendall (1979;as cited in Solomon et al., 2000). 


Frequency reading self-chosen 
books outside of school 


One item rated on a five-point scale that was developed for the purposes of the study reported by Battistich et al. (1989). This item assesses the frequency and 
enjoyment of reading books outside school. 


Frequency reading self-chosen books in 
school 


One item rated on a five-point scale that was developed for the purposes of the study reported by Battistich et al. (1989). This item assesses the frequency and 
enjoyment of reading books inside school. 



1 . This appendix presents a brief description of all outcomes that are presented in Appendices A3.1 -A4.3. 
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Appendix A3.1 Summary of study findings included in the rating for the behavior domain 1 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 
(students/ 
schools) 


Caring School 

Community Comparison 

group group 


Mean difference 4 
(Caring School 
Community - 
comparison) 


Statistical 
significance 6 
Effect size 5 (at a = 0.05) 


Improvement 

index 7 



The San Ramon Study (randomized controlled trial with confounding problems) 



Negative behavior 


Grades K-4 


350/6 


51.33 

(10) 


48.36 

(10) 


2.97 


0.29 


ns 


+12 


Spontaneous prosocial behavior 


Grades K-4 


350/6 


53.36 

(10) 


45.85 

(10) 


7.51 


0.74 


Statistically 

significant 


+27 


Harmoniousness 


Grades K-4 


350/6 


50.43 

(10) 


49.47 

(10) 


0.96 


0.09 


ns 


+4 


Supportive, friendly, and 
helpful behavior 


Grades K-4 


350/6 


52.35 

(10) 


47.11 

(10) 


5.24 


0.52 


Statistically 

significant 


+20 


Social competence 


Grade 4 


295/6 


1.66 

(0.46) 


1.62 

(0.47) 


0.04 


0.09 


ns 


+3 


Average 8 for behavior (The San Ramon Study) 










0.35 


ns 


+14 








The Six-District Study (quasi-experimental design) 








Altruistic behavior 


Grades 3-6 


1,986/10 


0.04 

(0.63) 


-0.03 

(0.66) 


0.07 


0.11 


ns 


+4 


Use of alcohol 


Grades 5-6 


635/10 


0.27 

(0.39) 


0.29 

(0.40) 


0.02 


0.05 


ns 


+2 


Use of marijuana 


Grades 5-6 


635/10 


0.04 

(0.22) 


0.07 

(0.28) 


0.03 


0.12 


ns 


+5 


Use of cigarettes 


Grades 5-6 


635/10 


0.13 

(0.30) 


0.12 

(0.32) 


-0.01 


-0.03 


ns 


-1 


Ran away from home 


Grades 5-6 


635/10 


0.09 

(0.30) 


0.10 

(0.34) 


0.01 


0.03 


ns 


+1 


Skipped school 


Grades 5-6 


635/10 


0.16 

(0.38) 


0.15 

(0.37) 


-0.01 


-0.03 


ns 


-1 


Damaged property on purpose 


Grades 5-6 


635/10 


0.23 

(0.45) 


0.27 

(0.45) 


0.04 


0.09 


ns 


+4 
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Appendix A3.1 Summary of study findings included in the rating for the behavior domain (continued) 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 
(students/ 
schools) 


Caring School 

Community Comparison 

group group 


Mean difference 4 
(Caring School 
Community - 
comparison) 


Statistical 
significance 6 
Effect size 5 (at a = 0.05) 


Improvement 

index 7 



The Six-District Study (quasi-experimental design) (continued) 



Stolen money or property 


Grades 5-6 


635/10 


0.26 

(0.43) 


0.25 

(0.45) 


-0.01 


-0.02 


ns 


-1 


Carried a knife, gun, 
or other weapon 


Grades 5-6 


635/10 


0.24 

(0.49) 


0.23 

(0.45) 


-0.01 


-0.02 


ns 


-1 


Threatened to hurt someone 


Grades 5-6 


635/10 


0.45 

(0.56) 


0.43 

(0.57) 


-0.02 


-0.04 


ns 


-1 


Hurt someone on purpose 


Grades 5-6 


635/10 


0.40 

(0.54) 


0.37 

(0.55) 


-0.03 


-0.06 


ns 


-2 


Taken a car without permission 


Grades 5-6 


635/10 


0.03 

(0.21) 


0.09 

(0.34) 


0.06 


0.22 


ns 


+9 


Been in a gang fight 


Grades 5-6 


635/10 


0.10 

(0.35) 


0.14 

(0.40) 


0.04 


0.11 


ns 


+4 


Thrown objects at people 


Grades 5-6 


635/10 


0.22 

(0.44) 


0.23 

(0.47) 


0.01 


0.02 


ns 


+1 


Been made fun of or 
called names 


Grades 5-6 


635/10 


0.98 

(0.54) 


0.99 

(0.57) 


0.01 


0.02 


ns 


+1 


Had property damaged 
on purpose 


Grades 5-6 


635/10 


0.44 

(0.52) 


0.56 

(0.56) 


0.12 


0.22 


ns 


+9 


Had property stolen from desk 


Grades 5-6 


635/10 


0.49 

(0.54) 


0.67 

(0.61) 


0.18 


0.31 


ns 


+12 


Had money or property 
taken by force or threat 


Grades 5-6 


635/10 


0.17 

(0.40) 


0.22 

(0.49) 


0.05 


0.11 


ns 


+4 


Been threatened with harm 


Grades 5-6 


635/10 


0.45 

(0.55) 


0.47 

(0.57) 


0.02 


0.04 


ns 


+1 


Been physically attacked 


Grades 5-6 


635/10 


0.28 

(0.47) 


0.34 

(0.53) 


0.06 


0.12 


ns 


+5 
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Appendix A3.1 Summary of study findings included in the rating for the behavior domain (continued) 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 
(students/ 
schools) 


Caring School 

Community Comparison 

group group 


Mean difference 4 
(Caring School 
Community - 
comparison) 


Statistical 
significance 6 
Effect size 5 (at a = 0.05) 


Improvement 

index 7 



Average 8 for behavior (The Six-District Study) 


0.07 


ns 


+3 


Domain average 8 for behavior across all studies 


0.21 


ns 


+8 



ns = not statistically significant 



1 . This appendix reports findings considered for the effectiveness rating and the improvement index. For the San Ramon Study, outcomes in the behavior domain were reported by the study authors for the first cohort only. For the purposes 
of this review, where findings were reported for multiple points in time for the same sample, only the most recent posttests with an eligible design were reviewed. For the Six-District Study, the findings pertain to outcomes after three 
years of program implementation. 

2. The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The standard 
deviations for the four behavior outcomes reported by Battistich et al. (1989) were estimated given that the authors used T-scores standardized within each grade level. A T-score is based on a normal distribution with a mean of 50 and 
standard deviation of 10. The findings included in this domain were eligible for review based on matching of the schools at baseline. 

3. Follow-up findings for the study reported by Battistich et al. (1989) were reported by Battistich (2003) for fifth-grade students using peer nominations of negative and positive behaviors. These outcomes were not included in this review 
because of attrition of schools in the follow-up years. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The direction of the mean difference was reversed for all outcomes except for altruistic behavior in the 
Battistich et al. (2000) study, so that a positive difference is associated with a decreased frequency of problem behavior. 

5. For an explanation of the effect size calculation, please see the Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. The level of statistical significance was reported by the study authors or, where 
necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See the Technical Details 
of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of both Battistich et al. (1989) and Battistich et al. (2000), corrections for clustering and multiple comparisons were 
needed, so the statistical significance reported by the WWC may differ from that reported by the authors. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

8. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A3.2 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 1 









Author’s findings from the study 2 














Mean outcome 
(standard deviation 3 ) 




WWC calculations 




Outcome measure 4 


Study 

sample 5 


Sample size 
(students/ 
schools) 


Caring School 

Community Comparison 

group group 


Mean difference 6 
(Caring School 
Community - 
comparison) 


Statistical 
significance 8 
Effect size 7 (at a = 0.05) 


Improvement 

index 9 



The San Ramon Study (randomized controlled trial with confounding problems) 



Democratic values 


Grade 4 


294/6 


3.31 

(0.37) 


3.12 

(0.45) 


0.19 


0.46 


ns 


+18 


Perceptual benevolence 


Grade 3 
(cohort 1) 


256/6 


1.79 

(0.19) 


1.75 

(0.24) 


0.04 


0.17 


ns 


+7 


Empathy 


Grade 4 
(cohort 1) 


256/6 


1.69 

(0.47) 


1.67 

(0.56) 


0.02 


0.04 


ns 


+3 


Concern for others 


Grade 4 
(cohort 1) 


294/6 


2.01 

(1.30) 


2.06 

(1.23) 


-0.05 


-0.04 


ns 


-4 


Concern for equality 


Grade 4 
(cohort 1) 


294/6 


3.19 

(1.19) 


2.92 

(1.36) 


0.27 


0.21 


ns 


+8 


Motive to help others learn 


Grade 4 
(cohort 1) 


294/6 


2.62 

(0.92) 


2.64 

(0.92) 


-0.02 


-0.02 


ns 


-1 


Social understanding 


Grade 4 
(cohort 1) 


317/6 


2.99 

(0.42) 


2.93 

(0.38) 


0.06 


0.15 


ns 


+6 


Social problem solving interview 


Grade 1 
(cohort 2) 


295/6 


2.21 

(na) 


2.11 

(na) 


0.10 


0.14 


ns 


+6 


Conflict resolution interview 


Kindergarten 
(cohort 2) 


318/6 


1.24 

(na) 


1.08 

(na) 


0.16 


0.18 


ns 


+7 


Social problem solving interview 


Grades 1 and 3 
(cohort 1) 


191/6 


2.58 

(na) 


2.44 

(na) 


0.15 


0.27 


ns 


+11 


Conflict resolution interview 


Grades K, 2, and 4 
(cohort 1) 


133/6 


1.36 

(na) 


1.16 

(na) 


0.20 


0.43 


ns 


+17 


Total self-esteem 


Grade 4 
(cohort 1) 


294/6 


2.35 

(0.80) 


2.40 

(0.89) 


-0.05 


-0.06 


ns 


-2 


Sense of community 


Grade 4 


294/6 


1.54 

(0.34) 


1.53 

(0.38) 


0.15 


0.39 


ns 


+15 


Average 10 for knowledge, attitudes, and values (The San Ramon Study) 








0.18 


ns 


+7 



(continued) 
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